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In recent years, a new approach for assessing the effects of student aid has 
emerged that can be used to evaluate the effects of aid strategies in colleges 
and universities. This article illustrates how these new models can be used to 
assess the effects of changes in government and institutional aid policy on 
persistence. This case study explores the implications of an increased reliance 
on loans in institutional aid packaging. 

R ecent developments in the application of logistic regression models for 
institutional research on student aid have improved the capability of 
institutions to assess their aid strategies on first-time enrollment and 
persistence. First, St. John (1992) proposed a workable approach to institu¬ 
tional research, then Somers (1992, 1995) tested and refined the methods for 
assessing the effects of aid. More recently, Somers and St. John (1997b) have 
illustrated how institutional comparisons can be used to assess diverse aid 
strategies across institutions. These models also can be adapted to evaluate the 
effects of changes in government and institutional bid policies. 

This paper illustrates how cross-year comparisons (assessing the effects 
of aid in successive years) can be used to assess the effects of change in aid 
packaging at a single campus. It adapts the workable persistence model (St. 
John, 1992; Somers and St. John, 1997a) to examine the effects of aid packages 
and aid thresholds 1 on within-year persistence by two cohorts of students who 
enrolled in a private college in Washington. In 1993-94, the State of Washing¬ 
ton expanded its state grant program. The same year, there was an expansion 
in eligibility for federal loans, due to changes in enabling legislation for student 
financial aid. 

In 1995, the Washington Higher Education Coordinating Board (HECfi) 
sponsored a comprehensive study on the effects of the increase in state grants. 
The study found that the increase in state grants actually improved persistence 
(Lee and St. John, 1996). This paper uses two years of private college data, 
collected as part of the Washington grant study, to illustrate how cross-year 
comparisons can be used to evaluate the effects of aid strategies. 


Statistical Methods The statistical methods, model specifics, and study limitations follow. 

This study uses logistic regressions, an approach that is especially well 
suited for research on persistence (St. John, Kirshstein and Noell 1991; 
Cabrera, 1994). The beta coefficients in the logistic models are converted to 
change in probability measures (delta p’s) using a method recommended by 
Petersen (1984). In addition, the baseline probability and percent of cases 
correctly predicted are presented. 2 

Three levels of statistical significance are reported for predictor variables. 
However, when an entire population is used (i.e., all enrolled sophomores and 
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Model Specifications 


juniors were used in this analysis, rather than a sample of this population) 
statistical significance of predictor variables does not have its normal meaning. 
Therefore, in a technical sense, it is no longer necessary to project significance 
from a sample to a population. However, these measures do provide indicators 
of whether measured effects are meaningful (have strong association), even if 
the term “significance” does not have its usual meaning. 

This study was limited to sophomores and juniors at the private college. 
College grades were available only for this study. 3 Because freshmen did not 
have college GPA’s, they were excluded from these analyses. Further, seniors 
were excluded because information on graduation was not available. 

The basic workable model (St. John, 1992) included variables related to 
background, achievement, college experience, and student aid. The variables 
related to each of these factors are described below. 

Ten variables were used for family background (see Table 1): males 
(coded as 1) were compared to females (coded as 0); African Americans and 
Hispanics were compared to other-race students (mostly Whites) in a set of 
design variables; married aid applicants (coded as 1) were compared to 
nonmarried aid applicants and other students (both coded as 0); dependent aid 
applicants and independent aid applicants were compared to non-aid applicants 
in a set of design variables 4 ; and aid applicants with low incomes, middle 
incomes, and high incomes were compared to nonaid applicants in a design 
set. 

Three variables related to achievement were included. Students college 
grades were recorded into a set of design variables: students with less-than-C 
averages, C averages (2.0 to 2.5), and A averages (above 3.5) were compared 
to students with B averages (above 2.5 to 3.5). This set of variables was a 
general indicator of achievement and provided sufficient information for our 
purposes, as discussed under Study Limitations below. 

Two variables related to college experience were included in the model: 
aid applicants who attended full-time were compared to other students; and 
sophomores were compared to juniors. These variables not only provided a 
basis for differentiating among different types of college experiences, but also 
controlled for the effects of year in college and type of college attended when 
assessing the effects of student aid. 

Two different approaches to assessing the effects of student aid were used 
in the analysis: aid packages and aid thresholds. First, a set of design variables 
was created that compared students with each type of aid package awarded to 
students who did not receive aid. This approach has been used previously (i.e., 
Somers, 1995). The analyses also illustrate the use of aid thresholds. In these 
analyses, students with low-, mid-, and high-range grants are compared to 
students who did not receive grants; students with low-, mid-, and high-range 
loans are compared to students who did not receive loans, and students with 
low-, mid-, and high-range work study awards are compared to students who 
did not receive grants. 5 
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TABLE 1 

Variable Coding for Private Four-Year 


VARIABLE Coding 

FAMILY BACKGROUND 

Male 1/0 

Age Years Old 

Ethnicity Design Set (Compares to other race, mostly white) 

African American 1/0 

Hispanic 1/0 

Married (Compares married aid applicants to others) 

Married 1/0 

Dependency Status (For aid applicants) 

Dependent 1/0 

Independent 1/0 

Family Income (Compares aid applicants to non-aid applicants) 

Low Income (lowest third) 1/0 

Mid Income (middle third) 1/0 

High Income (highest third 1/0 

ACHIEVEMENT 

Grades (Compares to students with B averages) 

A average (above 3.5 GPA) 1/0 

C average (2.5 to 2.0 GPA) 1/0 

Less than C (below 2.0 GPA) 1/0 


Analyses 


COLLEGE EXPERIENCE 

Full Time (Compares full time aid applicants to others) 

Full-Time .... 1/0 

Year in College (Compares to juniors) 

Sophomores 1/0 


FINANCIAL AID (Alternative Specifications) 

Aid Packages (Compares Aided Students to Non Aided Students) 

Grant Only 1/0 

Loan Only 1/0 

Loan/Work 1/0 

Grant/Loan 1/0 

Grant/ Work 1/0 

All Three 1/0 

Aid Thresholds 

Grants (Compares grant recipients to non-grant recipients) 

Low (lowest third) 1/0 

Mid (middle third) 1/0 

High (highest third) 1/0 

Loans (Compares loan recipients to non-loan recipients) 

Low (lowest third) 1/0 

Mid (middle third) 1/0 

High (highest third) 1/0 

Work (Compares work-study recipients to non-work-study recipients) 

Low (lowest third) 1/0 

Mid (middle third) 1/0 

High (highest third) 1/0 


JOURNAL OF STUDENT FINANCIAL AID 


9 



Study Limitations 


Private University 
Student Findings 


There are few limitations that merit consideration by the readers. First, these 
workable models do not include the measures of social and academic integra¬ 
tion frequently included in persistence research (Pascarella and Terenzini, 

1991) . However, this does not have an appreciable influence on our ability to 
measure the direct effects of student aid. 

Second, the lack of information on high school achievement limited the 
number of cases we could consider. Normally, it is recommended that institu¬ 
tions include high school achievement data from college applications (St. John, 

1992) . The adaptations made here minimize the negative effects of this missing 
information. 

Third, these cohorts had some missing values. Virtually all of the missing 
cases were attributable to missing grade point averages. It was assumed these 
values were randomly distributed, an assumption that was both necessary and 
reasonable. 

First, the descriptive statistics for the three cohorts are compared. Then the 
logistic analyses of the two cohorts are discussed. 

The undergraduates in the private university represent a relatively homo¬ 
geneous population (see Table 2). A large percentage of the private university 
cohorts applied for student aid (about 62% of each cohort), although those who 
applied for student aid represent relatively higher incomes than aid applicants 
in public colleges in Washington (Lee and St. John, 1996). Students in the 
lower-income portion of the aid-applicant population were from families that 
earned about twenty thousand dollars: the threshold was around $17,600 for 
the 1992 and $21,106 for the 1993 cohorts. The upper-income threshold was 
also higher: it was over $48,000 for the 1993 cohort. The costs of attending 
were also substantially higher at the private university which is the primary 
reason why more students applied for student aid. 

The ethnic composition of the private university changed slightly during 
the two-year period. The percentage that were Hispanics and African Ameri¬ 
cans dropped, indicating that fewer students from these ethnic backgrounds 
had the choice to attend this private college. The decline in the percentage of 
minority enrollment coincides with an increase in the percentage of aid 
packages comprised of student loans. However, it is not possible to identify 
causes for the change in percent of minority students, these changes in 
percentages could result from a flow through of students (i.e. freshmen and 
seniors are not included) as well as attrition. In fact, from the descriptive data 
we do not know whether these factors contributed to minority students if they 
dropped out. 6 

The private university substantially increased its emphasis on loans in 
1993. In the fall of 1992, 45.5% received grants in their aid packages, while 
51.0% received loans in their aid packages. In 1993,46.2% received grants and 
55.4% received loans. The percentage of students with loans increased more 
than the percentage with grants. Additionally, the average loan award also 
increased substantially faster than the average grant award. The average grant 
increased from $6,027 in fall 1992 to $6,251 in fall 1993. The increase in the 
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VARIABLE 

1992 

1993 

BACKGROUND 

Cohort 

Cohort 

Male 

39.6% 

43.3% 

Age 

24 yrs. 

23 yrs. 

Ethnicity 

African American 

1.5% 

0.8% 

Hispanic 

4.2% 

3.7% 

Married 

6.2% 

8.7% 

Dependency Status 

Dependent 

39.9% 

42.1% 

Independent 

22.8% 

22.2% 

Other (no aid applicant info.) 

37.3% 

35.7% 

Family Income: Percent of Cohort 

Low Income (Lowest third) 

19.9% 

19.6% 

Mid Income (Middle third) 

20.0% 

20.9% 

High Income (Highest third) 

20.6% 

21.3% 

Other (no aid applicant info.) 

39.5% 

38.2% 

Family Income: Threshold Level 

Low Income (Lowest third) 

<$17,600 

<$21,560 

High Income (Highest third) 

>$45,472 

>$48,401 

ACHIEVEMENT 

Grades 

A average 

17.3% 

19.6% 

B average 

55.8% 

56.7% 

C average 

17.1% 

16.2% 

Less Than C 

4.0% 

3.4% 

COLLEGE EXPERIENCE 

Full-Time 

54.7% 

56.3% 

Year in College 

Sophomore 

31.9% 

24.7% 

Junior 

68.1% 

75.3% 

FINANCIAL AID 

Aid Packages 

Grant Only 

3.4% 

2.6% 

Loan Only 

6.6% 

9.3% 

Work Only 

0.5% 

0.0% 

Loan/Work 

4.8% 

4.7% 

Grant/Loan 

13.6% 

12.5% 

Grant/Work 

2.5% 

2.2% 

All Three 

26.0% 

28.9% 

Aid Amount: Average for Awardees 

Grant $ 

$6,027 

$6,251 

Loan $ 

$7,057 

$8,585 

Work $ 

$2,515 

$2,788 

Number of Sample 

649 

623 
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The 1992 Cohort 


number of grant awards was partially attributable to an expansion in state 
grants. In contrast, the average loan soared from $7,057 to $8,585. This total, 
which includes loans from all sources, represents a relatively high average loan 
for a private college. 

The private university, like most private colleges in the United States, was 
priced too high to realize any gains in Pell awards when it raised its tuition, 
which means that increases in grants will come from either institutional funds 
or state funds if there is no growth in federal grant dollars. Usually this means 
that a substantial portion of tuition revenue increases are recycled into student 
aid (Hauptman, 1990a, 1990b). In this institution, there was a modest increase 
in state grant funding in 1993-94 (Lee and St. John, 1996). 

However, eligibility and award levels for federal loans did increase in the 
fall of 1993, as a result of the Reauthorization of the Higher Education Act. 
Accordingly, the average loan award at the private university climbed substan¬ 
tially in the fall of 1993, increasing by more than fifteen hundred dollars 
($1,500) over the prior year. This represents a substantial shift in the compo¬ 
sition of aid packages. 

The bottom line is that there appears to have been a reduction in opportu¬ 
nity for Hispanic and African American students at this one private university, 
due to a combination of factors. Rising tuition, the growth in federal loan 
programs, and the slow increase in grant dollars may have contributed to this 
decline in opportunity for minority students to attend this university. However, 
from descriptive data, we can not reach a conclusion about the relationship 
between changes in aid and changes in the percentage of minority enrolled in 
either cohort. 

The logistic analysis of the 1992 cohort in the private university is presented 
in Table 3. 

• Family Background : No family background variables were significant. 
This is not surprising given the homogeneity of the population. 

• Achievement. Students with less-than-C grades were less likely to persist 
(by about eleven percentage points). This is consistent with other persist¬ 
ence research (Pascarella and Terenzini, 1991). 

• College Experience : Attending full-time was significant. Full-time aid 
applicants were about ten percentage points more likely to persist. 

• Financial Aid: There was no change in influence on student aid variables 
for the 1992 cohort compared to the 1991 cohort. All types of aid 
packages continued to have a neutral association with persistence, indi¬ 
cating that in general aid remained adequate. 

The analysis of the effects on aid thresholds indicates that students with 
middle and high grants were less likely to persist in 1992 than students who 
did not receive grants. Grant aid was apparently not sufficient for students with 
mid- and high-range grants. This situation may have contributed to die drop in 
minority students in 1993-94. 

In combination these findings suggest that the private college successfully 
balanced aid packages in 1992, using other forms of aid to compensate for 
grants. This conclusion is based on the fact that aid packages were not 
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The 1993 Cohort 


7 

statistically significant, which means that packages were adequate. Thus, 
while grants per se were insufficient for some students, the overall aid packages 
were sufficient. 

The analysis of the effects of student aid on persistence by the 1993 cohort 
(Table 4), when compared to the analyses of the prior cohorts (Table 3), 
indicates that changes in the combination of government financial aid policy 
and institutional aid packaging had an influence on persistence in this private 
university. First we consider changes in the influence of other factors, com¬ 
pared to the prior cohorts, then we examine the effects of student aid. 

• Family Background: Age was significant and negative for the first time 
for the 1993 cohort. Each year of age differential reduced the probability 
of persistence by about .55 percentage points; each decade of age differ¬ 
ential reduced this probability by about 5.5 percentage points. 

Could the emergence of age as a significant variable be related to changes 
in student aid policy and packaging? There are reasons to suspect that there is 
such a linkage. Recall that the 1993 cohort had substantially larger average 
loans than the 1992 cohort. Also prior research indicates that adults are more 
sensitive to college costs than are traditional-college-age students (St. John and 
Starkey, 1995). 

It also appears that adults were more sensitive to the amounts of loans 
awarded and that this sensitivity is related to years of age. Apparently older 
students are more likely to ponder the implications of educational debt for their 
future. By increasing the emphasis it placed on loans, this private university 
may have decreased its probability of retaining older students. The juxtaposi¬ 
tion of the analyses of the 1992 and 1993 cohorts supports such a conclusion. 

• Achievement. Receiving less-than-C grades was negatively associated 
with within-year persistence again for the 1993 cohort. Having less-than- 
C grades decreased the probability of persistence by about 6 percentage 
points. This finding is consistent with the analyses of persistence by the 
1992 cohort (Table 4). 

• College Experience: Sophomores were less likely to persist in this analy¬ 
sis (Table 4). This is an interesting development. Perhaps sophomores 
were more likely to react negatively to the increased loan burden, a 
conclusion related to the findings on student aid. 

• Financial Aid: The analyses of the effects of student aid packages for 
the 1993 cohort also reveals that changes in aid packaging and amounts 
in fall of 1993, compared to the prior years, had an influence on persist¬ 
ence. When the different types of aid packages were considered, one type 
of aid package was slightly significant (alpha .1) and negatively associ¬ 
ated with persistence. Students who received packages with loans and 
work study were about 8.0 percentage points less likely to persist than 
non-aid recipients. This means that the awarding of self-help packages in 
fall of 1993 had become problematic for some students, possibly because 
of the increase in the amount of loans. However all other types of aid 
packages were not significant, indicating that most aid recipients had the 
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FAMILY BACKGROUND 

Packages Sig. 

Delta-p 

Thresholds 

Delta-p 

Sig. 

Male 

-0.00115 

-0.00386 


Age 

-0.00226 

-0.00210 


Ethnicity 

African American 

0.07478 

0.07161 


Hispanic 

0.11031 

0.11133 


Married 

0.03378 

0.03034 


Dependency Status 

Dependent 

-0.06515 

-0.05341 


Independent 

-0.05883 

-0.04113 


Family Income 

Low Income 

0.00516 

0.00631 


Mid Income 

0.03441 

0.03360 


High Income 

0.02495 

0.02397 


ACHIEVEMENT 

Grades * 

A average 

-0.04107 

-0.03958 


C average 

-0.00338 

-0.00072 


Less Than C 

-0.11347 .01 

-0.11266 

.01 

COLLEGE EXPERIENCE 

Full-Time 

0.09345 .1 

0.10957 

.01 

Sophomore 

-0.02813 

-0.02225 


FINANCIAL AID 

Aid Packages 

Grant Only 

Loan Only 

Loan/Work 

0.01429 

0.05582 

-0.03066 



Grant/Loan 

Grant/Work 

All Three 

-0.06358 

-0.07692 

-0.09189 



Aid Thresholds 

Grants 

Low 


-0.04934 


Mid 


-0.09338 

.05 

High 

Loans Low 

Mid 

High 


-0.08567 

-0.03561 

-0.02407 

-0.05799 

.1 

Work Study 

Low 

Mid 

High 


-0.06083 

-0.01418 

-0.03504 


MODEL INDICATORS 

N of Cases (D+) 

649(21) 

649 (24) 


Baseline P 

85.362 

85.362 


Percent Correlation Predicted 

85.67 

85.82 



"The design variables compare students in each category to otherwise average students with 
B averages. 
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FAMILY BACKGROUND 

Packages 


Thresholds 



Delta-p 

Sig. 

Delta-p 

Sig. 

Male 

0.03524 


0.03665 

.1 

Age 

-0.00550 

.01 

-0.00544 

.01 

Ethnicity 





African American 

-0.03786 


-0.03856 


Hispanic 

-0.04360 


-0.04741 


Married 

0.02290 


0.01512 


Dependency Status 





Dependent 

0.03944 


0.02719 


Independent 

0.04065 


0.04921 


Family Income 





Low Income 

0.02727 


0.03694 


Mid Income 

0.04404 


0.06117 


High Income 

0.02765 


0.05332 


ACHIEVEMENT 





Grades * 





A average 

-0.00229 


0.00263 


C average 

-0.00960 


-0.00810 


Less Than C 

-0.06119 

.05 

-0.5809 

.1 

COLLEGE EXPERIENCE 





Full-Time 

-0.00400 


0.01823 


Sophomore 

-0.06242 

.01 

-0.06329 


FINANCIAL AID 





Aid Packages 





Grant Only 

0.09057 




Loan Only 

-0.06649 




Loan/Work 

-0.07957 

.1 



Grant/Loan 

-0.06035 




Grant/Work 

0.09021 




All Three 

-0.06756 




Aid Thresholds 





Grants 





Low 



0.01398 


Mid 



0.03053 


High 



0.05058 


Loans 





Low 



-0.08063 

.05 

Mid 



-0.07252 

.1 

High 



-0.08129 

.05 

Work 





Low 



0.00144 


Mid 



-0.04284 


High 



-0.02339 


MODEL INDICATORS 





N of Cases 

618 


618 


Baseline P 

90.85 


90.85 


Percent Correlation Predicted 

90.78 


90.78 



* The design variables compare students in each category to otherwise average students with 
B averages. 
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Conclusion 


same probability of persisting when family backgrounds, achievement, 
and college experience were taken into account. 

In the analysis of the effects of aid thresholds in the private university, all 
levels of loans were negatively associated with persistence. In combination 
with the other finding that students who received packages with loans and work 
study were less likely to persist, this finding suggests that loans had become 
problematic for students enrolled in the private university. The packages 
analysis and the supplemental analysis of aid thresholds point to a potentially 
troubling development. Both findings indicate the increased use of self-help 
aid had been problematic for students in 1993-94. 

However, it is also apparent that the increase in grants helped the institu¬ 
tion overcome the problem with grant aid, for the 1992 cohort grants were 
insufficient for students with either moderate or large grants. However, for the 
1993 cohort, grants were adequate, indicating the increase in state grants helped 
mitigate this problem. 


There is some good news embedded in the comparison of persistence across 
the two years. The baseline probability of persistence improved. For the 1992 
cohort, the baseline probability was 85.4% and for the 1993 cohort it was 
90.1%, indicating a substantial improvement in the annual retention rate. 
Further, since the analyses of the effects of student aid are relative to the 
baseline—because students with certain types of aid are compared to students 
who do not have aid that same year—and the baseline probability improved in 
1993-94, the negative findings for 1993-94 are not as problematic as they may 
seem at first. It appears that increases in grants and loans helped, given that 
grants were insufficient in 1992-93, but were sufficient in 1993-94. In addition, 
other forces at the university, forces outside the variable set used here, also 
could help explain the improvement in persistence rates. For example, the 
college could have developed a successful retention program. 

However, there is a need to be concerned about loan burden. Federal 
student aid now emphasizes loans more than grants. Further, most private 
colleges not only surpass the limits of costs embedded in federal grant schemes, 
but have made substantial investments of their own resources in need-based 
grants. The increased federal loan limits in 1993 provided institutions with an 
opportunity to change the tuition grant spiral by increasing the emphasis on 
loans. However, there are apparent costs associated with this strategy. In this 
case study there is evidence that loans had become burdensome. Thus private 
colleges are faced with tough choices when they make annual decisions about 
tuition charges and internal allocations to aid. Perhaps more private institutions 
need to consider alternative ways of structuring student aid and prices, such as 
tuition reduction, an approach used by some private colleges (Hamm, 1995; 
Rothman, 1995). 

This study illustrates that the new workable models can be used to assess 
the effects of changes in aid policies from one year to the next. Such techniques 
can help aid administrators and institutional researchers assess their financial 
aid strategies. Further annual analyses of this type can help you systematically 
evaluate the effects of change in your aid policy. 
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Endnotes 

1. The thresholds approach was developed in response to the Washington HECB (1994) request for proposals. It has not 

previously been used in published research on the effects of students financial aid. 

2. In the original study, four other indicators of model strength, in addition to the percent of cases correctly predicted, were 

used. Readers can refer to the full study (St. John and Starkey, 1996) if they wish to view these detailed statistics. 

3. In the original model (St. John, 1992) and in the initial test of the model (Somers, 1992, 1995) both high school and 

college grades were available for the persistence analyses. Since high school and college grades tend to be highly 
correlated, the fact that only college grades were available did not prohibit us from completing the study. 

4. Dependency status was needed as a distinct variable because family income was used for dependent aid applicants and 

student income was used for independent aid applicants. 

5. In each of these design sets, students receiving these types of aid were divided into three, nearly equal categories, then 

compared to students who did not receive aid (St. John and Starkey, 1995; St. John and Starkey, 1996). 

6. However, our persistence analysis for 1992 (Table 3) indicates grants were insufficient and therefore aid policies could 

have contributed to this situation. 

7. This interpretation of neutral coefficients has been confirmed by other recent studies (e.g. St. John, Paulsen and Starkey, 

1996). 
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